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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address « 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )E3 Responsive to communication(s) filed on 20 August 2001 . 
2a)D This action is FINAL. 2b)|3 This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) K Claim(s) 1^9 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) 13 Claim(s) 1-3.5,6 and 9 is/are rejected. 

7) |3 Claim(s) 4. 7 and 8 is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) E3 The specification is objected to by the Examiner. 

10) 13 The drawing(s) filed on 27 February 2001 is/are: a)D accepted or b)|3 objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) 13 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
aM AH b)D Some * c)D None of: 

1 .|3 Certified copies of the priority documents have been received. 

2.Q Certified copies of the priority documents have been received in Application No. . 



3.D Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
See the attached detailed Office action for a list of the certified copies not received. 
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3) □ Information Disclosure Statement(s) (PTO-1449 or PTO/SB/08) 5 > □ Notice of lnformal Patent Application (PTO-1 52) 
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DETAILED ACTION 
Drawings 

1 . The drawings are objected to as failing to comply with 37 CFR 1 .84(p)(5) 
because they do not include the following reference sign(s) mentioned in the 
description: Page 10, line 7 - reference sign "501". A proposed drawing 
correction or corrected drawings are required in reply to the Office action to avoid 
abandonment of the application. The objection to the drawings will not be held in 
abeyance. 

2. The drawings are objected to as failing to comply with 37 CFR 1 .84(p)(5) 
because they include the following figure not mentioned in the description: Fig. 
32. A proposed drawing correction, corrected drawings, or amendment to the 
specification to add the reference sign(s) in the description, are required in reply 
to the Office action to avoid abandonment of the application. The objection to the 
drawings will not be held in abeyance. 

3. The drawings are objected to as failing to comply with 37 CFR 1 .84(p)(5) 
because they include the following reference sign(s) not mentioned in the 
description: reference characters VPEN-TIP, VPEN-FRM, VPEN, TIP-SW, 
POINT-PREF-BTN, STD-PTN, MIN-DIST, PTN-DIC, PROC-TAB, PROC-TAB, 
PROC-COUNT, PROC-TAB-ELM, TIP-FIELD, PROC-FIELD, ACT-COUNT, 
ACT-TAB-ELM, PTN-ID2, ACT-FIELD, ACT-TAB, IP-AREA, MSG-AREA, PB- 
AREA; which are all found throughout several figures. A proposed drawing 
correction, corrected drawings, or amendment to the specification to add the 
reference sign(s) in the description, are required in reply to the Office action to 
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avoid abandonment of the application. The objection to the drawings will not be 
held in abeyance. 



Specification 

4. The disclosure is objected to because of the following informalities: 
a. Page 9, line 6: change 11 penholder 103" to -penholder 201--. 
Appropriate correction is required. 



Claim Rejections - 35 USC § 102 

5. The following is a quotation of the appropriate paragraphs of 35 

U.S.C. 102 that form the basis for the rejections under this section made in this 

Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in 
public use or on sale in this country, more than one year prior to the date of application for patent in 
the United States. 

6. Claims 1-3 are rejected under 35 U.S.C. 102(b) as being anticipated by 
Taguchi et al. (U.S. Patent 5,774,602). 

Regarding claims 1 and 2, Taguchi et al. discloses in Fig. 18 a pen 
type input device with a camera including a pen (86) and a camera (71 ) 
mounting on the pen (86) and picking-up an image of a penpoint, wherein 
the camera (71 ) is mounted on the pen (86) so that a center position of the 
image picked up by the camera (71 ) is located at left or right side of the tip 
end of the pen (86) (col. 6, lines 20-22; col. 14, lines 16-24). Since the 
camera as disclosed by Taguchi et al. is removable one could position the 
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camera anywhere on the pen, whether it be on the right side or the left 
side of the pen depends on the like of the user. 

Regarding claim 3, Taguchi et al. discloses in Fig. 18 a pen type 
device with a camera (71) wherein the camera (71) is mounted on the pen 
(86) so that the camera (71 ) may pivot relative to the penholder within a 
range at least 0 to 90° (col. 6, lines 20-22; col. 14, lines 16-24). It is 
inherent that if the camera is removable then the camera may pivot 
relative to the penholder within a range of at least 0 to 90°. 
7. Claims 5, 6, and 9 are rejected under 35 U.S.C. 102(b) as being 
anticipated by Toshifumi et al. ("PaperLink: A Technique for Hyperlinking from 
Real Paper to Electronic Content"). 

Regarding claim 5, Toshifumi et al. discloses a pen type input 
device with a camera including a pen, a camera mounting on the pen and 
picking-up an image of a penpoint, and an information processing 
apparatus performing process of an image picked up the camera, wherein 
the information processing apparatus extracts an object to be processed 
from an image picked up by the camera, determines a process to be 
executed from the picked-up image, and performs process of the 
extracted object (sections: The Videopen and Interaction, pages 328-330; 
Figs. 2-7). 

As interpreted by the Examiner: As can be seen from Figs. 2-4 
Toshifumi et al. discloses a camera mounted on a pen, wherein the 
camera picks up an image of the penpoint. Furthermore, after the image 
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is taken the information processing apparatus extracts an object to be 
processed (either a word or a highlighted word) from the picked-up image, 
determines a process to be executed from the picked up image (if the 
object is a highlighted word (page 328-329), the information processing 
apparatus establishes a hyperlink; and if the object is only a piece of text 
(page 329), the information processing apparatus classifies the text as 
data and not a hyperlink), and performs a process of the extracted data (if 
the object was highlighted text then a hyperlink is created and if the object 
was only text then the text is considered data and stored in a buffer for 
later use). 

Regarding claim 6, Toshifumi et al. discloses a pen type input 
device with a camera wherein the information processing apparatus 
determines a process to be executed on the basis of a color picked up by 
the camera (sections: The Videopen and Interaction, pages 328-330; Figs. 
2-7; section: Implementation Details - Structural Overview, page 330). 
Toshifumi et al. discloses that if a colored region is found around the 
center of the image, the segmentation unit extracts the region; otherwise, 
the unit extracts the black and white pattern corresponding to the text 
itself. Toshifumi et al. also discloses that if there is a highlighted region 
then a hyperlink will be established and if there is no highlighted region 
then the object extracted is regarded as data and stored in a buffer for 
later use. 
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Regarding claim 9, Toshifumi et al. discloses a pen type input 
device with a camera including a pen, a camera mounting on the pen and 
picking-up an image of a penpoint, and an information processing 
apparatus performing process of an image picked up by the camera, 
wherein the information processing apparatus extracts a standard pattern 
picked up by the camera and extracts an object by correcting an image 
picked up on the basis of a result of extraction (section: Implementation 
Details - Segmentation and Pattern Recognition, pages 330-331 ). 

As interpreted by the Examiner: As can be seen from Figs. 2-4 
Toshifumi et al. discloses a camera mounted on a pen, wherein the 
camera picks up an image of the penpoint. Furthermore, after the image 
is taken the information processing apparatus extracts a standard pattern 
(the highlighted region) from the image and extracts an object by 
correcting an image picked up on the basis of a result of extraction (the 
highlighted region is corrected by passing the highlighted region through 
an additional filter and the object extracted is the text that was 
highlighted). 



Allowable Subject Matter 

8. Claims 4 and 7-8 are allowed. 

9. The following is an examiner's statement of reasons for allowance: prior 
art fails to teach or fairly suggest a pen type input device with a camera including 
a pen, a camera mounting on the pen and picking-up an image of a penpoint, 
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and an information processing apparatus performing process of an image picked 
up by the camera, wherein.... 

a the information processing apparatus makes judgment of an 

orientation of an objective image picked up by the camera depending 
upon a position of a tip end of the pen in the image picked-up by the 
camera (claim 4). 

b the information processing apparatus detects position of the 

object extracted from the image picked by the camera and the penpoint to 
display an image indicating pointing of the pen on the basis of the result of 
detection (claims 7 and 8). 

Any comments considered necessary by applicant must be submitted no 
later than the payment of the issue fee and, to avoid processing delays, should 
preferably accompany the issue fee. Such submissions should be clearly labeled 
"Comments on Statement of Reasons for Allowance." 

Conclusion 

10. The prior art made of record and not relied upon is considered pertinent to 

applicant's disclosure. 

a. Tano et al. (U.S. Patent 5,850,058) discloses a pen with a camera 
mounted to it used in combination with a tablet includes an image 
reconfiguring unit for composing, after correcting image distortion 
generated due to inclination of the pen body for each image in the vicinity 
of a pen head which is photographed along with the strokes of the input 
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pen by a CCD camera, the series of image into of images into a general 



Any inquiry concerning this communication or earlier communications from 
the examiner should be directed to Heather R Long whose telephone number is 
703-305-0681. The examiner can normally be reached on Mon.-Thurs.: 7:00 am 
- 4:30 pm, and every other Fri.: 7:00 am - 3:30 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, Andrew Christensen can be reached on (703) 308-9644. 
The fax phone number for the organization where this application or proceeding 
is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from 
the Patent Application Information Retrieval (PAIR) system. Status information 
for published applications may be obtained from either Private PAIR or Public 
PAIR. Status information for unpublished applications is available through 
Private PAIR only. For more information about the PAIR system, see http://pair- 
direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll- 
free). 
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